1. Field of the Invention
The present invention relates to an image processing unit which can correct blurs and noise in an image including a moving subject by image synthesis. In particular, it relates to an image processing unit, an image processing method, and an image processing program which can accurately detect a motion in images to be synthesized with different brightness and generate images with good tone representation.
2. Description of the Prior Art
There are known digital cameras having various functions, taking advantage of high-speed operation of an image sensor such as CMOS. For example, one of them can continuously record five or more images per second or another one can shoot moving pictures at 1,000 fps or more. Such digital cameras adopt a dynamic range expansion technique in which dark to light portions of images captured with different exposure are synthesized at an arbitrary mix ratio to generate good images in expanded tone.
Meanwhile, another type of known digital camera includes an image processing unit to synthesize continuous images for the purpose of removing blurs due to camera shakes or noise in images captured with a long time exposure and preventing the occurrence of ghosts. By image synthesis, it is able to average random noise and reduce noise due to camera shakes or shooting in a dark condition.
To synthesize continuous images, it is necessary to align the positions of the images, the position of one image as a reference image with those of other images as comparative images. For synthesizing images including a completely still subject, the position alignment can be simply conducted by making the sizes of the images the same. However, for synthesizing images including a moving subject, it is necessary to first determine a direction in which the subject is moving and align the positions of the images in accordance with the motion of the subject. Without the motion of the subject taken into account, image synthesis will increase blur and noise in the images. In view of the above, Japanese Patent No. 3935500, for example, discloses an image processing unit which determines the motion of a subject before image synthesis and synthesizes images while aligning the positions of the images according to information on the subject's motion.
By combining the dynamic range expansion with the subject's motion detection and position alignment, it is made possible to effectively prevent blurs and ghosts in images, generate high quality images and improve the usability of users.
However, the dynamic range expansion does not include the position alignment needed for the images including a moving subject and is used for images with different brightness or exposure. Since the position alignment in the above document can be conducted only on images with the same brightness, it is very difficult to synthesize images with different brightness by combining the above processings, aiming for blur correction.